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Abstract 

Background: There has been increasing emphasis on evidence-based approaches to improve patient outcomes 
through rigorous, standardised and well-validated approaches. Clinical guidelines drive this process and are largely 
developed based on the findings of systematic reviews (SRs). This paper presents a discussion of the SR process in 
providing decisive information to shape and guide clinical practice, using a purpose-built review database: the 
Cochrane reviews; and focussing on a highly prevalent medical condition: hypertension. 

Methods: We searched the Cochrane database and identified 25 relevant SRs incorporating 443 clinical trials. 
Reviews with the terms 'blood pressure' or 'hypertension' in the title were included. Once selected for inclusion, the 
abstracts were assessed independently by two authors for their capacity to inform and influence clinical decision- 
making. The inclusions were independently audited by a third author. 

Results: Of the 25 SRs that formed the sample, 12 provided conclusive findings to inform a particular treatment 
pathway. The evidence-based approaches offer the promise of assisting clinical decision-making through clarity, but 
in the case of management of blood pressure, half of the SRs in our sample highlight gaps in evidence and 
methodological limitations. Thirteen reviews were inconclusive, and eight, including four of the 12 conclusive SRs, 
noted the lack of adequate reporting of potential adverse effects or incidence of harm. 

Conclusions: These findings emphasise the importance of distillation, interpretation and synthesis of information to 
assist clinicians. This study questions the utility of evidence-based approaches as a uni-dimensional approach to 
improving clinical care and underscores the importance of standardised approaches to include adverse events, 
incidence of harm, patient's needs and preferences and clinician's expertise and discretion. 

Keywords: Systematic review, Research-in-practice, Research implementation, Translational research, Evidence-based 
practice, Clinical decision-making 



Background 

Much has been written about evidence-based medicine 
(EBM) and its potential to inform practice. The termin- 
ology surrounding the aggregation of evidence in a sys- 
tematic way implies that there is a practical import to this 
work, and the terms "evidence-based practice" (EBP) and 
"evidence-based approaches" (EBA) have arisen from this 
concept. The EBA are among the most significant con- 
temporary ways of conceptualising health care, both the- 
oretically and practically [1], and are viewed by advocates 
as the favoured means of integrating research findings into 
clinical practice [2] . 
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Despite the appeal of EBA, and the widespread recog- 
nition of this approach in driving care, they present a 
number of challenges, and there has been a steady 
undercurrent of concern about their pervasive accept- 
ance and promotion. Debate has focussed on several key 
factors, including the nature of evidence, particularly the 
privileging of some forms of evidence and the marginalising 
and devaluing of others [2-5]. Concerns that the domin- 
ation of the EBA has (and will continue to) foster 'a very 
rational, traditional, biomedical approach to research use/ 
evidence-based practice' [5] forms a crucial plank of the 
critique. Additional major critiques of EBA highlight 
concerns that they do not meet the challenges of rare 
diseases or unusual presentations, and may foster a "cook- 
book" approach to practice [6]. An additional view is that 
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they may potentially have a deleterious effect on patient/ 
physician relationships; including the charge that they cre- 
ate or contribute to an environment of paternalism and 
reductionism that can render patient preference and ex- 
perience invisible and irrelevant [1-3,7]. According to this 
critique, EBA can foster a therapeutic milieu in which 
patients are positioned as passive recipients in the medical 
encounter [1], and clinicians are limited in their ability to 
draw on intuitive and other forms of knowledge [7]. 

Notwithstanding the theoretical critique however, the 
aim of EBA is to facilitate the integration of the best 
available research evidence into practice by transparently 
assimilating information and aggregating data in ways 
that can help guide clinical decision-making [6]. Indeed, 
the EBA are attractive because they promote the idea of 
research-in-practice, yet present an alternative to each 
clinician locating, reading and assimilating the plethora 
of contemporary research literature on any clinical issue 
[6]. However, we argue that in some areas, even where a 
substantial body of literature exists, there is a lack of evi- 
dence to demonstrate that the EBA in fact provide the 
information necessary to effectively guide clinicians. 

Purpose of this paper 

The purpose of this paper is to explore the efficacy of 
the systematic review (SR) process in providing de- 
cisive information to shape and guide clinical practice. 
To do this we examined SRs on the management of 
blood pressure. We chose this because hypertension is 
a well-researched, common and persistent condition 
with a significant effect on health outcomes [8]. The 
Cochrane reviews on reducing blood pressure became 
the lens through which we explored the efficacy of the 
SR process in providing contemporary and clear infor- 
mation upon which clinical practice decisions could be 
made. 

Methods 

Independent manual searches of the reviews published 
on the Cochrane Database of Systematic Reviews (http:// 
www.cochrane.org/cochrane-reviews) on 1/12/11 were 
undertaken to search for SRs on interventions to lower 
blood pressure. Reviews with the terms 'blood pressure' 
or 'hypertension' in the title were included. Once se- 
lected for inclusion, the abstracts were assessed inde- 
pendently by two authors. SRs were excluded if they 
pertained to blood pressure targets, pulmonary hyper- 
tension, hypertensive emergencies, acute cardiac events, 
gestational, postpartum, infant, neonatal and child hyper- 
tension. Furthermore, SRs were also excluded if the 
details of the number of clinical trials were not provided, 
and if the SR did not include any completed studies. 
Inclusions were independently audited by a third author. 



Results 

Twenty-five SRs representing a combined total of 443 
clinical trials were identified (see Table 1). Of these SRs, 
the majority reported pharmaceutical-based interven- 
tions. Salt intake, weight-reducing diets, timing of medi- 
cation and relaxation therapies were each the subject of 
a single SR. Twelve SRs made conclusive assertions, that 
is, they used language that denoted a degree of certainty. 
The remaining SRs (n = 13) were inconclusive (see 
Table 2). 

Discussion 

In eight SRs, including four of the 12 SRs that were 
considered to be conclusive, the lack of adequate re- 
porting of potential adverse effects or incidence of harm 
was specifically noted (see Table 2). These factors would 
exert an influence on clinicians and on the confidence 
they can have in selecting specific treatments as a result 
of any SR. There has long been concern that clinicians 
can be slow to take up research findings into their prac- 
tice, with a common focus being on accessibility of 
current information and the most efficacious means of 
dissemination [9,10]. However, findings from this exer- 
cise suggest there needs to be more of an onus on 
researchers to produce findings that are able to inform 
practice in a more useful way. 

Attention is also drawn to the use and effect of extrane- 
ous words such as 'appears to' in reaching conclusions. It 
could be seen that use of this ambiguous language means 
that results are being presented cautiously but it could 
also have the effect of creating doubt or suggest an unwill- 
ingness or inability to have an opinion either way. Lan- 
guage is a problematic area in research [11], but much of 
the scrutiny about language and how it is used to present 
research focuses on the use of jargon or exclusionary lan- 
guage. When considering the use of ambiguous language - 
language which creates doubt and jeopardises meaning, 
questions about the nature of language and how it is used 
arise. How essential is the word (that causes the doubt) to 
the significance and meaning of the message? If we took 
those words out, would we have clear cut messages? If the 
words create the doubt, what is the point in having this 
process? If results cannot be interpreted with confidence, 
how can conclusions be reached through the SR process? 

The inconclusive nature of several SRs raises a number 
of additional issues for consideration. The point of EBA 
as a systematic transparent process is to allow for the 
aggregation of data so that statistical significance may be 
achieved. In our sample, it was noted that some of the 
included studies were unreliable because of potential 
bias, questionable rigour or other factors which meant 
studies were deemed to be poor quality. The EBA have 
been the catalyst for the development of a ranking sys- 
tem, known as the hierarchy of evidence [12,13]. 
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Table 1 Included systematic reviews 



Title 



Date Authors 



No of 
trials 



Creatine and creatine analogues in hypertension and cardiovascular disease 
Evening versus morning dosing regimen drug therapy for hypertension 

Long-term effects of weight-reducing diets in hypertensive patients 

Blood pressure lowering efficacy of beta-blockers as second-line therapy for primary hypertension 
Spironolactone for hypertension 

Blood pressure lowering efficacy of potassium-sparing diuretics (that block the epithelial sodium channel) for 
primary hypertension 

Pharmacotherapy for hypertension in the elderly 

Blood pressure lowering efficacy of alpha blockers for primary hypertension 

Blood pressure lowering efficacy of angiotensin converting enzyme (ACE) inhibitors for primary hypertension 

Blood pressure lowering efficacy of angiotensin receptor blockers for primary hypertension 

Blood pressure lowering efficacy of coenzyme Q10 for primary hypertension 

Blood pressure lowering efficacy of diuretics as second-line therapy for primary hypertension 

Blood pressure lowering efficacy of loop diuretics for primary hypertension 

Blood pressure lowering efficacy of reserpine for primary hypertension 

Blood pressure lowering in patients without prior cerebrovascular disease for prevention of cognitive impairment 
and dementia 

Methyldopa for primary hypertension 

Blood pressure lowering efficacy of renin inhibitors for primary hypertension 
First-line drugs for hypertension 

Relaxation therapies for the management of primary hypertension in adults 
Potassium supplementation for the management of primary hypertension in adults 
Magnesium supplementation for the management of primary hypertension in adults 
Calcium supplementation for the management of primary hypertension in adults 

Beta-blockers for hypertension 

Combined calcium, magnesium and potassium supplementation for the management of primary hypertension in 
adults 

Effect of longer-term modest salt reduction on blood pressure 
TOTAL 



2011 
2011 

2011 

2010 
2010 

2010 

2010 
2009 
2009 
2009 
2009 
2009 
2009 

2009 

2009 

2009 
2009 

2009 

2009 
2009 
2009 
2009 

2009 
2009 
2008 



Horjus, et al. 


I I 


Zhao P, et al. 


21 


Siebenhofer, 


8 


et al. 




Chen, et al. 


20 


Batterink, et al. 


5 


Heran, et al. 


6 


Musini, et al. 


15 


Heran, et al. 


10 


Heran, et al. 


92 


Heran, et al. 


46 


Ho, et al. 


3 


Chen, et al. 


56 


Musini, et al. 


9 


Shamon & 




Perez 


4 


McGuinness, 


4 


et al. 




IVldll, CI dl. 


1 2 


Musini, et al. 


6 


Wright & 


z'-l 


Musini 


Dickinson, et al. 


25 


Dickinson, et al. 


5 


Dickinson, et al. 


12 


Dickinson, et al. 


13 


Wiysonge, 


13 


et al. 




Beyer, et al. 


3 


He & 




MacGregor 


20 



443 



Darlenski et al. [6] make the point that good clinicians 
will be able to draw on EBA, and we concur with this, 
even where findings are inclusive. Furthermore, it is 
recognised and understood that information drawn from 
EBA is only one of a number of factors that inform clin- 
ical judgement. But, clearly there is a need to provide 
some guidance about interventions that may never be 
able to be proven in the current evidentiary way. At the 
moment this is left to institutions through mechanisms 
such as clinical pathways, regulators such as the Na- 
tional Institute of Clinical Excellence (NICE), and the 
National Health and Medical Research Council. Some 



progress is being made as groups both in the community 
and within organisations are trying to develop method- 
ologies to assist this process, such as through modelling. 

Systematic reviews are only as good as the sum of the 
parts. Strategies such as standardised reporting of clin- 
ical trials through CONSORT methods offer some pro- 
mise in increasing the rigour and usefulness of research 
(http://www.consort-statement.org/). The recognition of 
complex interventions in clinical care is another import- 
ant consideration [14]. In this instance some interven- 
tions may have high internal validity, yet less external 
validity. 
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Table 2 Authors' conclusions 



Comments from author's conclusions (all comments taken directly from author conclusions as appeared on Cochrane 
Database of Systematic Reviews 1/12/11) 



Authors 



This review found inconclusive evidence to decide on the use of creatine analogues in clinical practice. 
The effect on death and the size of myocardial infarction and the heart function were unclear. 

Given the small sample size of the discussed trials and the heterogeneity of the population included in these reports, larger clinical 
studies are needed to confirm these observations. 

In terms of BP lowering efficacy, for 24-hour SBP and DBP, the data suggests that better blood pressure control was achieved with 
bedtime dosing than morning administration of antihypertensive medication, the clinical significance of which is not known. 

The magnitude of the effects are uncertain as a result of the small number of patients and studies that could be included in the 
analyses. 

It is not known whether weight loss reduces mortality and morbidity. 
No useful information on adverse effects was reported in the relevant trials. 

In conclusion, there is no evidence for effects of weight loss diets on death or long-term complications and adverse events. 

In addition, results on blood pressure and body weight could be considered uncertain, because not all studies were included in the 
analyses. 

The different effect on diastolic BP means that beta-blockers have little or no effect on pulse pressure whereas thiazides cause a 
significant dose-related decrease in pulse pressure. 

From the limited available evidence, spironolactone appears to lower blood pressure compared to placebo to a similar degree in 
patients with primary (essential) hypertension when doses of 100-500 mg/day are given. 

Batterink, et al. A dose of 25 mg/day did not statistically significantly reduce systolic or diastolic blood pressure, compared to placebo. 

Given the lack of a dose-response, coupled with a possible increased risk in adverse events with higher doses, doses of 25 to 100 
mg/day are reasonable.There is no evidence of the effect of spironolactone on clinical outcomes in hypertensive patients. 



Horjus, et al. 



Zhao P, et al. 



Siebenhofer, 
et al. 



Chen, et al. 



ENaC blockers do not have a statistically or clinically significant BP lowering effect at low doses but trials at higher doses are not 
Heran, et al. available. 

The review did not provide a good estimate of the incidence of harms associated with ENaC blockers 

Thirteen RCTs were found and these trials suggested that first-line beta-blockers for elevated blood pressure were not as good at 
Wiysonge, et al. decreasing mortality and morbidity as other classes of drugs: thiazides, calcium channel blockers, and renin angiotensin system 
inhibitors. 

The BP lowering effect of alpha blockers is modest; the estimate of the magnitude of trough BP lowering of -8/-5 mmHg is likely 
an overestimate. 

Heran, et al. There are no clinically meaningful BP lowering differences between different alpha blockers. 

The review did not provide a good estimate of the incidence of harms associated with alpha blockers because of the short 
duration of the trials and the lack of reporting of adverse effects in many of the trials. 



Heran, et al. 



Heran, et al. 



Ho, et al. 



Chen, et al. 



Musini, et al. 



There are no clinically meaningful BP lowering differences between different ACE inhibitors. 
The BP lowering effect of ACE inhibitors is modest. 

The review did not provide a good estimate of the incidence of harms associated with ACE inhibitors because of the short duration 
of the trials and the lack of reporting of adverse effects in many of the trials 

The evidence from this review suggests that there are no clinically meaningful BP lowering differences between available ARBs. 
The BP lowering effect of ARBs is modest and similar to ACE inhibitors as a class. 

The review did not provide a good estimate of the incidence of harms associated with ARBs because of the short duration of the 
trials and the lack of reporting of adverse effects in many of the trials 

Due to the possible unreliability of some of the included studies, it is uncertain whether or not coenzyme QI0 reduces blood pressure in 
the long-term management of primary hypertension 

Thiazides when given as a second-line drug have a dose related effect to lower blood pressure that is similar to when they are 
added as a first-line drug. This means that the BP lowering effect of thiazides is additive. 

Because of the short duration of the trials and lack of reporting of adverse events, this review does not provide a good estimate of 
the incidence of adverse effects of diuretics given as a second-line drug 

Based on the limited number of published RCTs, the SBP/DBP lowering effect of loop diuretics is modest . . . and is likely an overestimate 
due to the high risk of bias in the included studies. 

There is no clinically meaningful BP lowering differences between different drugs within the loop diuretic class. 
The dose ranging effects of loop diuretics could not be evaluated. 



Saltman et al. BMC Medical Informatics and Decision Making 201 3, 1 3:25 
http://www.biomedcentral.com/1472-6947/13/25 



Page 5 of 7 



Table 2 Authors' conclusions (Continued) 



The review did not provide a good estimate of the incidence of harms associated because of the short duration of the trials and 
the lack of reporting of adverse effects in many of the trials. 

Reserpine is effective in reducing SBP roughly to the same degree as other first-line antihypertensive drugs. 

However, we could not make definite conclusions regarding the dose-response pattern because of the small number of included trials. 



Shamon & 
Perez 



McGuinness, 
et al. 



Mah, et al. 



Musini, et al. 



Wright & 
Musini 



More RCTs are needed to assess the effects of reserpine on blood pressure and to determine the dose-related safety profile before 
the role of this drug in the treatment of primary hypertension can be established 

There is no convincing evidence from the trials identified that blood pressure lowering in late-life prevents the development of 
dementia or cognitive impairment in hypertensive patients with no apparent prior cerebrovascular disease. 

There were significant problems identified with analysing the data, however, due to the number of patients lost to follow-up and the 
number of placebo patients who received active treatment. This introduced bias. 

More robust results may be obtained by conducting a meta-analysis using individual patient data. 

Methyldopa lowers blood pressure to varying degrees compared to placebo for patients with primary hypertension. 

to effect on clinical outcomes, however, remains uncertain. 

Overall reporting of adverse effects was poor so no conclusions can be drawn about the adverse effect profile. 

This meta-analysis shows that methyldopa reduces systolic/diastolic blood pressure by approximately 13/8 mmHg compared to 
placebo. 

Aliskiren has a dose-related blood pressure lowering effect better than placebo. 
This effect is similar to that determined for ACE inhibitors and ARBs 
First-line low-dose thiazides reduce all morbidity and mortality outcomes. 

First-line ACE inhibitors and calcium channel blockers may be similarly effective but the evidence is less robust. 

In view of the poor quality of included trials and unexplained variation between trials, the evidence in favour of causal association 
between relaxation and blood pressure reduction is weak. 

Dickinson, et al. It was difficult to disentangle their effects, especially as many trials used a combination of methods 

some of the reduction in blood pressure was almost certainly due to aspects of treatment that were not related to relaxation, such as 
frequent contact with professionals who were trying to help. 

Due to small number of participants in the two high quality trials, the short duration of follow-up, and the unexplained heterogeneity 
between trials, the evidence about the effect of potassium supplementation on blood pressure is not conclusive. 

Further high quality RCTs of longer duration are required to clarify whether potassium supplementation can reduce blood pressure 
and improve health outcomes. 

Most included trials were of poor quality, so their results may not be reliable. 
Dickinson, et al. j ne tr j a j s were not j on g en0U g n or j ar g e en0 ugh to measure whether potassium supplements reduce the risk of death, heart attack or 
stroke, which may be caused by high blood pressure. 

This review does not confirm whether potassium supplements can lower high blood pressure and therefore does not recommend 
them for treating hypertension. 

More trials enrolling a large number of participants with long periods of follow-up are necessary to know whether or not potassium 
supplements can lower high blood pressure. 

In view of the poor quality of included trials and the heterogeneity between trials, the evidence in favour of a causal association between 
magnesium supplementation and blood pressure reduction is weak and is probably due to bias. 

Dickinson, et al. This is because poor quality studies generally tend to over-estimate the effects of treatment. 

Larger, longer duration and better quality double-blind placebo controlled trials are needed to assess the effect of magnesium 
supplementation on blood pressure and cardiovascular outcomes. 

Due to poor quality of included trials and heterogeneity between trials, the evidence in favour of causal association between calcium 
supplementation and blood pressure reduction is weak and is probably due to bias. 

Dickinson, et al. This is because poor quality studies generally tend to over-estimate the effects of treatment. 

Larger, longer duration and better quality double-blind placebo controlled trials are needed to assess the effect of calcium 
supplementation on blood pressure and cardiovascular outcomes. 

Treating healthy persons (60 years or older) with moderate to severe systolic and/or diastolic hypertension reduces all cause 
Musini, et al. mortality and cardiovascular morbidity and mortality. The decrease in all cause mortality was limited to persons 60 to 80 years of 
age. 
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Table 2 Authors' conclusions (Continued) 





None of the trials were of high quality, so their results may not be reliable. 




Beyer, et al. 


We found no robust evidence that supplements of any combination of potassium, magnesium 
BP in adults. 


or calcium reduce mortality, morbidity or 




More trials are needed to investigate whether the combination of potassium & magnesium 


is effective. 


He & 


Our meta-analysis demonstrates that a modest reduction in salt intake for a duration of 4 or more weeks has a significant and, from 
a population viewpoint, important effect on blood pressure. 


MacGregor 


These results support other evidence suggesting that a modest and long-term reduction in 
strokes, heart attacks, and heart failure. 


population salt intake could reduce 



Key 

Italicised text Inconclusive result. 

Underlined text No/poor reporting of adverse effects or incidences of harm. 



Conclusions 

Though the EBA offer the promise of direction through 
clarity, in the case of reducing blood pressure, and des- 
pite the enormous amount of research that has been 
undertaken, the clarity offers little direction for the fu- 
ture, in the main, aggregating the past. Rather, it 
highlights evidentiary gaps and weaknesses. Indeed, one 
of the real values of the SR process is that it summarises 
what we know, which is potentially not what we need to 
know - what should I do for the 85 year old women sit- 
ting in my clinic? 

Our results highlight a potential lack of efficacy of the 
SR process in driving future practice, which may leave fu- 
ture practice potentially open to other, less evidence-based 
directions. Krumholz [15] has recently highlighted the im- 
perative of method in generating clinical research that is 
able to effectively inform practice. Through this exercise 
we have shown that even in areas where it might be 
thought to be strong, such as in the aggregation of drug 
trials, the results were often not clear and therefore their 
utility for informing practice is compromised. This may be 
a factor of the analytic techniques that are used. To be 
useful in clinical practice, there is a need to develop more 
robust methodologies and to ensure that research pro- 
cesses are well described and sufficiently transparent to 
allow data aggregation to occur with confidence. 

Furthermore, it was noted that in some SRs inadequate 
information about prevalence of harm, complications and 
potential adverse effects were presented. In order that 
results are as clinically meaningful and useful as possible, 
there is a need to ensure reporting of presence or absence 
of these factors. 

This review also emphasises the need for expertise and 
interpretation when developing clinical practice guidelines, 
particularly in the absence of robust Level 1 evidence and 
differences in clinical trial populations and patient groups. 
Indeed, it is certainly important to identify where the evi- 
dence does not support a particular line of practice. We 
argue that the corollary is not disproven by this approach, 
that is, practice may be improved by a particular interven- 
tion despite the null hypothesis, but the evidence isn't 



strong enough in a normative way. There are many pos- 
sible reasons why the level of evidence to support EBA in 
practice will not always be strong enough to inform prac- 
tice. Assessing change in practice is difficult and robust 
methodologies have yet to be derived; the process is long- 
term and it will take some time for robust evidence to 
emerge; and, the evidence provided to support clinical 
decision-making may no longer be useful in practical 
settings. Based upon this review, data from SRs are only 
one part of the picture. Using systematic guideline develop- 
ment processes such as in ADAPTE framework are an im- 
portant part of this process (http://www.adapte.org/www/). 

Clearly there is a desire for contemporary and rigorous 
information upon which clinical treatment can be guided 
and individual clinical decisions made. The demand for 
this is evidenced in Cochrane's own usage data, which 
states that "every day someone, somewhere searches The 
Cochrane Library every second, reads an abstract every 
two seconds and downloads a full-text article every three 
seconds" (www.cochrane.org: accessed 16/1/12). 

Initially, the EBA have been viewed as a way of incorp- 
orating research into practice. The EBA does not always 
have high utility and reference for consistently assisting 
clinicians, particularly in groups not included in clinical 
trials. While the use of a single health issue may be 
viewed as a limitation of this paper; in using this com- 
mon health issue - blood pressure management - as a 
lens, we have raised questions about whether SRs as the 
core of clinical guideline development can reliably influ- 
ence practice in an interventional way, and about the 
strength of evidence that is available to inform clinical 
practice in some key areas. 
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